Regularizers versus Losses for Nonlinear Dimensionality Reduction
نویسندگان
چکیده
We demonstrate that almost all nonparametric dimensionality reduction methods can be expressed by a simple procedure: regularized loss minimization plus singular value truncation. By distinguishing the role of the loss and regularizer in such a process, we recover a factored perspective that reveals some gaps in the current literature. Beyond identifying a useful new loss for manifold unfolding, a key contribution is to derive new convex regularizers that combine distance maximization with rank reduction. These regularizers can be applied to any loss.
منابع مشابه
Regularizers versus Losses for Nonlinear Dimensionality Reduction: A Factored View with New Convex Relaxations
We demonstrate that almost all nonparametric dimensionality reduction methods can be expressed by a simple procedure: regularized loss minimization plus singular value truncation. By distinguishing the role of the loss and regularizer in such a process, we recover a factored perspective that reveals some gaps in the current literature. Beyond identifying a useful new loss for manifold unfolding...
متن کاملGlobal Versus Local Methods in Nonlinear Dimensionality Reduction
Recently proposed algorithms for nonlinear dimensionality reduction fall broadly into two categories which have different advantages and disadvantages: global (Isomap [1]), and local (Locally Linear Embedding [2], Laplacian Eigenmaps [3]). We present two variants of Isomap which combine the advantages of the global approach with what have previously been exclusive advantages of local methods: c...
متن کاملAn Empirical Comparison of Dimensionality Reduction Methods for Classifying Gene and Protein Expression Datasets
The recent explosion in availability of gene and protein expression data for cancer detection has necessitated the development of sophisticated machine learning tools for high dimensional data analysis. Previous attempts at gene expression analysis have typically used a linear dimensionality reduction method such as Principal Components Analysis (PCA). Linear dimensionality reduction methods do...
متن کاملLinear versus nonlinear dimensionality reduction for banks' credit rating prediction
Dimensionality reduction methods have shown their usefulness for both supervised and unsupervised tasks in a wide range of application domains. Several linear and nonlinear approaches have been proposed in order to derive meaningful low-dimensional representations of high-dimensional data. Among nonlinear algorithms manifold learning methods, such as isometric feature mapping (Isomap), have rec...
متن کاملNonlinear Dimensionality Reduction – Locally Linear Embedding versus Isomap
Real data of natural and social sciences is often very high-dimensional. However, the underlying structure can in many cases be described by a small number of features. Recently two new nonlinear methods for reducing the dimensionality of such data, Locally Linear Embedding and Isomap, have been suggested and successfully applied. This report compares both algorithms by means of several synthet...
متن کامل